| term | df | sumsq | meansq | statistic | p.value |
|---|---|---|---|---|---|
| setSize | 1 | 0.0003843 | 0.0003843 | 0.0292196 | 0.8642903 |
| iteration | 1 | 7.7448344 | 7.7448344 | 588.7892899 | 0.0000000 |
| setSize:iteration | 1 | 0.6490973 | 0.6490973 | 49.3466348 | 0.0000000 |
| Residuals | 1988 | 26.1498145 | 0.0131538 | NA | NA |
| term | df | sumsq | meansq | statistic | p.value |
|---|---|---|---|---|---|
| setSize | 1 | 0.1612980 | 0.1612980 | 15.49667 | 8.55e-05 |
| iteration | 1 | 9.5609337 | 9.5609337 | 918.56458 | 0.00e+00 |
| setSize:iteration | 1 | 0.7419769 | 0.7419769 | 71.28527 | 0.00e+00 |
| Residuals | 1988 | 20.6922155 | 0.0104086 | NA | NA |
The log-likelihood procedure resulted in more parameter-sets fitting participants over all:
log-likelihood: 50 unique parameter-sets (biased:36, LTM:20, meta_RL:2, RL:0)
RMSE: 37 unique parameter-sets (biased:11, LTM:15, meta_RL:9, RL:2)
According to the figure below it seems like the differences between the fits of the two models are very small with the RMSE method potentially being a better fit, especially for set size 6 data.
#>
#> Welch Two Sample t-test
#>
#> data: corr by method
#> t = 6.3954, df = 317.1, p-value = 5.735e-10
#> alternative hypothesis: true difference in means is not equal to 0
#> 95 percent confidence interval:
#> 0.04212052 0.07955197
#> sample estimates:
#> mean in group rmse mean in group logL
#> 0.9046639 0.8438277
| estimate | estimate1 | estimate2 | statistic | p.value | parameter | conf.low | conf.high | method | alternative |
|---|---|---|---|---|---|---|---|---|---|
| 0.0374 | 0.128 | 0.0905 | 8.61 | 0 | 149 | 0.0288 | 0.046 | Welch Two Sample t-test | two.sided |
| model | mean | median | sd | min | max |
|---|---|---|---|---|---|
| biased | 43.2459 | 8 | 119.99204 | 2 | 657 |
| LTM | 13.5000 | 9 | 15.20561 | 2 | 48 |
| metaRL | 3.0000 | 3 | 0.00000 | 3 | 3 |
These figures (specifically BIC fig 2) show that the best fit model has strong evidence against the second best fit model.These fits are better compared to the RMSE method (see RMSE document for comparison).
BIC fig 1
BIC Fig 2
BIC fig 3
| subjects | X1 | X2 | X3 | model |
|---|---|---|---|---|
| 6218 | 1.6800691 | 0.7662433 | 0.0131273 | LTM |
| 6226 | 20.7272476 | 0.1642479 | 1.5106469 | LTM |
| 6234 | 17.4765377 | 0.4880064 | 0.3893591 | LTM |
| 15005 | 26.7061902 | 6.8416407 | 8.8128153 | biased |
| 15006 | 14.9020846 | 17.2172782 | 3.6532177 | biased |
| 15007 | 26.5035268 | 0.2165939 | 5.6548364 | biased |
| 15012 | 22.6166963 | 3.6827696 | 6.3863653 | LTM |
| 15020 | 21.0249436 | 7.4761139 | 4.9440946 | LTM |
| 28307 | 4.3697057 | 2.1578692 | 3.2079443 | biased |
| 29245 | 0.0178342 | 7.1627899 | 3.3654804 | biased |
Figure 11.
#>
#> Welch Two Sample t-test
#>
#> data: estimate by setSize
#> t = 10.149, df = 142.26, p-value < 2.2e-16
#> alternative hypothesis: true difference in means is not equal to 0
#> 95 percent confidence interval:
#> 0.02799973 0.04154511
#> sample estimates:
#> mean in group s3 mean in group s6
#> 0.11481641 0.08004399
| setSize | type | model | mean | se |
|---|---|---|---|---|
| s3 | behav | biased | 0.1163739 | 0.0021718 |
| s3 | behav | LTM | 0.1130357 | 0.0036576 |
| s3 | behav | metaRL | 0.0851190 | 0.0017857 |
| s3 | model | biased | 0.1141015 | 0.0032627 |
| s3 | model | LTM | 0.1131286 | 0.0012347 |
| s3 | model | metaRL | 0.0973333 | 0.0065714 |
| s6 | behav | biased | 0.0792870 | 0.0036193 |
| s6 | behav | LTM | 0.0826984 | 0.0043388 |
| s6 | behav | metaRL | 0.0765873 | 0.0115079 |
| s6 | model | biased | 0.0849828 | 0.0022839 |
| s6 | model | LTM | 0.1022738 | 0.0017649 |
| s6 | model | metaRL | 0.0829286 | 0.0030714 |
Figure 12
| statistic | p.value | parameter | method |
|---|---|---|---|
| 193.4056 | 0 | 2 | Kruskal-Wallis rank sum test |
| statistic | p.value | parameter | method |
|---|---|---|---|
| 391.6311 | 0 | 2 | Kruskal-Wallis rank sum test |
| group1 | group2 | p.value |
|---|---|---|
| LTM | biased | 0 |
| metaRL | biased | 0 |
| metaRL | LTM | 0 |
| group1 | group2 | p.value |
|---|---|---|
| LTM | biased | 0 |
| metaRL | biased | 0 |
| metaRL | LTM | 0 |
Figure 13.
| term | df | sumsq | meansq | statistic | p.value |
|---|---|---|---|---|---|
| setSize | 1 | 5.5580884 | 5.5580884 | 438.4466841 | 0.0000000 |
| type | 1 | 0.1911125 | 0.1911125 | 15.0757993 | 0.0001050 |
| model | 2 | 1.0508771 | 0.5254386 | 41.4489263 | 0.0000000 |
| setSize:type | 1 | 0.0062083 | 0.0062083 | 0.4897418 | 0.4840835 |
| setSize:model | 2 | 0.9516216 | 0.4758108 | 37.5340689 | 0.0000000 |
| type:model | 2 | 0.0831864 | 0.0415932 | 3.2810574 | 0.0376904 |
| setSize:type:model | 2 | 0.0648987 | 0.0324494 | 2.5597497 | 0.0774516 |
| Residuals | 3972 | 50.3521358 | 0.0126768 | NA | NA |
| variable | mean | median |
|---|---|---|
| alpha | 0.1500000 | 0.1500000 |
| egs | 0.3000000 | 0.3000000 |
| bll | 0.5000000 | 0.5000000 |
| imag | 0.3000000 | 0.3000000 |
| ans | 0.3000000 | 0.3000000 |
| strtg | 0.4402444 | 0.3604394 |
Figure 15